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ABSTRACT 

A LISP program converts a mathematical expression stored 
in list structure form, into a text-book style visual display. 
To do this, requires the selection and positioning of the 
individual symbols which make up the expression, using a 
combination of global and local information. This memo describes 
a table-driven picture-compiler which makes the necessary 
information available. Syntax rules have been written for a large 
class of mathematical expressions. These rules are simplified 
by the introduction of concepts concerning the relative position 
of symbols. In addition to the symbols and their coordinates the 
program sends a parsing of the symbols to the display. This prograi 
is a refinement of the system proposed by M.L. Minsky in 
Artificial Intelligence Memo. 61, Mathscope; Part I. 
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I. Introduction 

This memo describes a LISP program which computes the 
pictorial representation of a mathematical expression from a LISP 
S-expression representation of the expression. Since the language of 
mathematics contains a number of special cases, the computation will 
be introduced with a summary of the results produced by each part 
of the program and the general principles followed. 

Because of the possible use of many transformations and 
identities, a mathematical expression can be represented in many 
equivalent forms. For instance a(b+c) could be represented as ab + ac. 
In certain contexts the picture of one of these forms may be easier to 
read than the picture of the others. For instance a - x might be 
preferred to a + (-1 • X). On the other hand, the use of several 
equivalent internal forms complicates the writing of routines needed 
to transform mathematical expressions. Furthermore, the most convenient 
internal form might be difficult to read. In the first pass of the 
picture compiler the internal form is transformed into an equivalent 
form which is easy to read. In addition, parentheses are added. 
The LISP expression resulting from this first pass has the form of a 
tree of subexpressions. Each node of the tree is a mathematical 
operation and the branches from that node are the arguments of that 
operation. At the ends of the branches are the individual variables 
and constants. The form of this internal tree is preserved throughout 
the remaining steps of the picture compilation. For example, the 
expression a = c • d + e has the tree: 




c - 'd 
Mathematical operations are represented in "pictures" 

by symbols placed in special arrangements. A classification of the 

representations will be given later. The situation is complicated 

by the fact that sometimes the same symbols and positional notations 

are used in different combinations to represent difrerenl maLliemat ica 1 

operations. The second pass of the picture compiler rewrites the 

internal tree in terms of these common symbols and positions. T1k> 

symbols and positions are expressed in terms of a set of special forms. 
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These forms are shown in Figure 1 through Figure 11. The compiler 
starts at the base of the internal tree and works out to the ends of 
the branches, rewriting the tree at each node. This rewriting intro- 
duces additional nodes into the internal tree, but these new nodes are 
marked so that'the old structure is preserved. The display resulting 
from a node and its branches is call a picture part . Sometimes the 
form of a picture part depends on the dimensions of its arguments. 
In this case the second and third pass must be applied to the argtmients 
before the second pass can be applied to this part. 

In the third pass each picture part is inscribed in a rec- 
tangle. The compiler starts at the ends of the branches of the inter- 
nal tree and works toward the base. The compiler first inscribes each 
individual constant and variable in a rectangle, defined by its width, 
height, and depth. A dimensioned rectangle is then chosen in turn for 
each larger picture part; i.e., operator with arguments. The dimensions 
of each rectangle depend on its own operator and also on the dimensions 
of those rectangles associated with the arguments of the picture part. 
In addition, we compute the relative positions of the arguments of each 
picture part with respect to the lower left hand corner of its cir- 
cumscribed rectangle. 

Once the entire expression has been dimensioned , it can be 
positioned on the oscilloscope face. The final pass of the compiler 
then sends to the display the name, size, and position of each symbol. 
These symbols are interspersed with non-displaying left and right pseudo- 
parentheses. These pseudo-parentheses group the picture symbols into a 
tree of sub-expressions identical with the internal tree structure obtained 
from pass one of the picture compiler. Thus, a light-pen reference m . 
picture symbol can be identified with th e smallest subexpression in tho 
picture tree which contains it, and with the c orresponding T.T SP subexpression 
in the internal-- tree . This makes it possible easily to designate mathe- 
matically meaningful picture-parts as arguments for other operations. 

^* Transformations to F acilitate Semantic Interpretation 

A method is needed for referencing meaningful subparts of 
the LISP source expression by pointing with a light pen to symbols 
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in the displayed picture expression. Thus, before the picture is com- 
piled, the source expression is transformed using mathematical relations 
into a form which makes it easy to establish a one-to-one mapping be- 
tween subparts of the picture expression and mathematically meaningful 
subparts of the LISP source expression. Then, in addition to the pic- 
ture symbols, the compiler sends the display a series of left and right 
pseudo-parentheses which parse the picture symbols into a tree identical 
with the tree formed by the transformed source expression. When the 
light pen is pointed at a picture symbol, the smallest picture sub- 
expression which contains it is intensified and the corresponding LISP 
expression may then be referenced. The source expression is transformed 
in order to bri«g it into a better pictorial form, for instance, the 
order of the product is reversed so that ( f^cdx) ' d can be written 
d • J cdx. 

Although the picture compiler is powerful enough to handle most 
all of the notations which arise in mathematics, compiler rules have been 
written for the mathematical expressions which arise in the solution 
of non-linear differential equiations. These expressions are represented 
by LISP S-expressions.* In brief, S-expressions are defined recursively 
as follows: Any number or string of alphanumeric characters is an 
S-expression. One or more S-expressions separated by spaces and 
surrounded by parentheses is an S-expression. Individual numbers and 
alphanumeric strings are called atoms . Individual variables and 
constants in the mathematical expressions are represented by atoms. 
The mathematical operators are represented by S-expressions consisting 
of the operator name, its arguments in some given order, and the atom 
NIL. NIL denotes the null S-expression; it is replaced by temporary 
results during the transformation of expressions and represents the 
property list of the mathematical operator. The individual mathematical 
operators currently in the program are: 

(Note: most of the operators can take any number of arguments, in the 
obvious manner.) 

1. (PLS ABC NIL ) ^ A + B + C 

2. (PRD A B C NIL ) :^ A • B • C 



* See LISP 1.5 Manual. 
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3. (FKT I J NIL) =: ~ 

4. (PWR A B NIL) -ES A^ 
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dA^dC^ 



5. (DRV A B C D E NIL) ^ d^^ 

c 

B 

6. (ITG D A B C NIL) =: C CdD 

C 

7 . (SUM A B C D NIL) = ^ D 

A=B 

(C=D 
A=B 
9. (NAM ABC NIL) ^ ^A,B 

10. (FAB NIL) ■=. F(A,B) 

11. (NAM A B (F C D NIL) NIL) =; F (C,D) 

A , B 

12. (FIL A NIL) ^ a: 

13. (ABS A NIL) ^ I a| 

Where A, B, C, D and E are arbitrary expressions, I and J are 
integers, and F is any atomic symbol not recognized as an operator. 
-X is represented by (PRD -1 X NIL) and i is represented by 
(PWR X -1 NIL). If one of the factors of a sum orporduct is a number, 
it is often the left-most argument. 

The first compiler pass performs the following transformations 
(PLS NAB NIL) — ^ (FLS A B N NIL) 
(DRV S 1 (F S NIDNIL) — ^ (DRV 1 1 (F S NIL)NIL) 

-N -> (NEC N NIL) 
(PRD -1 A NIL) ~> (NEC A NIL) 

(PRD A (SUM A B C D NIL) E NIL) ~> (PRD A E (SUM A B C D NIL)NIL) 
(PRD A (PWR B -1 NIL)NIL)— :> (DVD A B NIL) 

(PRD A (PWR B (PRD _1 C NIL)NIL)NIL) -^ (DVD A (PWR B C NIL)NIL) 
(FAB NIL) — 7" (FUNCTION FAB NIL) 
(PRD A (ITG A B C D NIL) E NIL) ^^ (prd A E (ITG A B C D NIL)NIL) 

X -^ (ATOM X NIL) 
(NAM X Y (F A B NIL)NIL) ~9^ (FUNCTION (NAM X Y F NIL) A B NIL) 
Where N is a number, X is a number or variable symbol, and S is a 
non-atomic subexpression 
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In addition to these transformations, levels of parentheses are 
inserted. For instance (PRD A (PLS B C NIL)NIL) — > (PRD A 
(PAREN (PLS B C NIL)NIL)NIL) . No parentheses are used around the 
arguments of FUNCTION, PAREN, ABS , EVL or NAM. A size cliange always 
removes the need for parentheses. In addition, the integration and 
summation symbols stand in place of parentheses for operators on their 
left and a transcendental function is parenthesized if it is raised to 
a power. Otherwise, parentheses are placed around an argument of an 
operator if the main operator of the argument has lox,7er precedence. 
Precedence is determined from the following list: PAREN, NAM, =, 
FUNCTION, FTL, PWR, FRT, PRD, =, DVD, ITG, DRV, NEC, - , PLS, =, SUM. 
Operators separated by = have equal precendence. 

The expression resulting from these transformations is saved as 
the internal tree for the later analysis of light pen references. For 

O A 

example, the expression Y = X + g would be represented by the S-expression 

(EGN (ATOM Y NIL) (PLS(PWR (ATOM X Nil) (ATOM 2 NIL)fJH. 

(DVD (ATOM A NIL) (ATOM B NIL)NIL)NIL)NIL) . It has the tree structure. 




II- Transformations to Facilitate Syntactic Construction 

To a very large extent, mathematical notation becomes, in the 
mind of the individual, a model of the abstract concepts he is manipula- 
ting. For instance, consider the operation of canceling terms from 
both sides of an equation or from the numerator and denominator of an 
indicated division, or the operation of bringing a term to the other 
side of an equation, or the operation of matrix multiplication. There- 
fore, the notation must have properties which make it easy to visualize. 
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Lack of generality makes mathematical notation easier to read. 
Different mathematical operators should be represented by different 
sorts of notation and the mathematical operator with the highest 
precedence should be represented by the most compact form. For 
example, compare the equivalent Boolean expressions (((A U B) U C) 
U (D E)) and A + B + C + DE. Furthermore, when an operation is 
applied to complex arguments, it should be possible to visualize 
it in the same manner as when it is applied to simple ones. For 
instance, the introduction of parentheses or other such fences 
allows (A + B)C to be visualized in the same manner as DC, with 
the operation expressed as concatenation. Sometimes a change of 
size is used to aid in visualizing a complex argument as a single 
unit. As a final example, consider the choice of f to represent 
the drivative when it is to be thought of as an operation of high 
precedence, but the choice of ^ for the derivative when its proper- 
ties as a ratio are to be used. 

Although it is difficult to make strict classifications, it 
is useful to distinguish seven distinct pictorial forms used in 
mathematical notation. As stated above, the more compact foniis are 
often used for operations of higher precedence with the exception 
that fences and size changes permit the use of complex arguments in 
the simpler forms; size changes allow complex forms to have high 
precedence . 

The forms in rough order of decreasing precedence, with some 
examples of each are: 

1. supersub tj , • X^ • ^t 

I'l • ^ '3 

2. concatenation with 

variable size fence in(x,y); |x| ; (b); (a) 

symbols and separa- 
tion s3mibols 

3. concatenation 2X • XYZ 
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b 6 

4. binding symbol — : f xdx ; ^ X 



x-3 



5 . hybrid 



d xdy 



6. concatenation with x=y ; a+b+c ; A B;A*B*C 
infix symbols 

7. title (El) X 

Since the same mechanisms, such as concatenation, are used 
in more than one of these seven forms, we will discuss the simpler 
ones first. 
1. Concatenation 

A rewrite rule may be associated with any operator. For 
example, if concatenation is used to represent multiplication then 
there might be a rule: 

(PRD A B NIL) -^ (CONCAT A B NIL) . 
The computation is simplified if concatenation is considered to be 
a binary operator. It is therefore necessary to have recursive 
rules such as : 

(PRD X^ X^ -— X^ NIL) -> (CONCAT X^ (CONCAT X - — (CONCAT 

^n-1 ^n^^^^ "-" NIL)NIL). 
This rule, however, makes a multiple level structure out of a single 
level structure and thus destroys the form of the internal tree 
which is needed to parse the symbols for the display. To remove this 
difficulty an indicator is put on the property list of the CONCAT 
form to show that it is not to be considered a node in the internal 
tree when the symbols are sent to the display by pass four. A 
pseudo-form, DELIMIT, is introduced to correspond to the old PRD 
operator. The shape of DELIMIT is just the shape of its argument. 
The rule above then becomes: 

(PRD X^ X^ — X^ NIL ) -> (DELIMIT(CONCAT X^ 

(CONCAT X^_^ X^ (UNDELIMIT)) — (UNDELIMIT)) (UNDELIMIT)) NIL). 
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^' ^^"^^t^^^tion with variable size fence .svmhm. and seo.r.tinn ..^k.i. 

In the example in(X,Y), the size of the comma does not depend 
on the size of the arguments X and Y so it can be handled by the 
form CONCAT. On the other hand, the size of the parentheses depends 
on the dimensions of the enclosed expression and so it is necessary 
to introduce a parenthesis form. The rule for functions is: 

(FUNCTION X^ X^ — X^ NIL) ^ (DELIMIT (CONCAT X^ (PAREN(CONCAT 
X^ (CONCAT(ATOM , (UNDELIMIT)) — (CONCAT(ATOM , (UNDELIMIT) 
X^ (UNDELIMIT)) (UNDELIMIT)) (UNDELIMIT)) (UNDELIMIT)) 
(UNDELIMIT) )NIL). 
The size of the parentheses is chosen during the dimension pass. 
They are considered to be symbols introduced by the PAREN fom, 
rather than arguments of the form. The dimension pass also tells 
the PAREN operator the depth of the parentheses contained in its 
argument. The PAREN operator then chooses parentheses, brackets, 
or braces accordingly. For example, the compiler would produce ' 
^a o Jc .. (b+d)| ^ . In the cases of transendental functions, 
the parentheses are ommitted if the argument is a single variable, 
a product of two variables, or a division. 
3. Binding symbol 

Often a size change is associated with a binding symbol. 
One might want to make the size change recursive; however, it takes 
only a few size changes to make the range of sizes too large. The 
current system uses characters which differ in size by a factor of 
two and in this case more than one size change is unsatisfactory. 
Fortunately, expressions occurring in practice would rarely require 
more than one size change if the recoursive rule were to be used. 
The second pass carries the size with it as it goes out the branches 
of the internal tree; the rewrite rule for the mathematical operator 
of any node of the tree may specify that certain branches from that 
node are to be rewritten using the smaller size. In order to 
present the rewrite rules, the smaller size has been indicated by a 
1 on the rewritten expressions property list. In actual fact, the 
size is saved on a pushdown list where it can be retrieved by the 
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third pass. An example of a rule involving a size change is the 
rule for integrals where the expressions for the limits are re- 
written using the smaller size: 

(ITG XI (ATOM A NIL) (ATOM B NIL) X2 NIL) -^ 
(ITG (CONCAT (ATOM D (UNDELIMIT)) XI (UNDELIMIT)) 
(ATOM A (1)) (ATOM B (1)) X2 NIL). 

4. Supersub 

This notation is used for exponentiation and for subscripting. 
The placement of subscripts is a property of the particular variable 
or function name. The subscripts can be placed at any of the four 
corners, so that there are 15 possible spatial arrangements. To 
avoid using 15 distinct but similar operators, null arguments have 
been introduced. They are treated as picture parts having zero 
dimensions. When the rewriting pass discovers a NAM operator it 
picks up a subscript-placement- list associated with the variable 
or function name in question. This is a list of the form (XI X2...Xn), 
where each Xi is one of the symbols NE, NW, SE, SW. The ith argument 
of NAM is then placed at corner Xi. Two or more arguments at the 
same corner are separated by commas. Placement of arguments must 
start at the NE corner and proceed counter-clockwise. When the 
subscript-placement- list is exhausted the remaining arguments are 
placed at the SE corner. For example if H has the subscript-place- 
ment-list (NW SW SW) then: 

(NAM XI X2 X3 X4 (ATOM H NIL) NIL-> (SUPERSUB NIL XI 
CONCAT X2 (CONCAT (ATOM , (UNDELIMIT)) X3 (UNDELIMIT)) 
(1 UNDELIMIT)) X4 (ATOM H NIL)NIL). 
If a subscripted variable name is raised to a power the exponent 
takes the NE position if there are no NE subscripts. The rewrite 
rule uses SUPERSUB and the operator PWRUP which preserves the original 
tree by removing the exponent from the SUPERSUB level during the 
dimension pass and using it for its second argument. 
For example: 

(PWR (NAM XI X2 NIL) X3 NIL-^ (PWRUP (SUPERSUB X3 NIL NIL 
XI X2 NIL) NIL NIL) . 
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Note that this combination of levels must be handled carelully if one 
desires to use a scheme where duplicate subexpressions are rewritten 
and dimensioned only once. 

5. Concatenation with infix symbols . 

This form has two possible spatial representations. It it is 
too wide for one line, then some of the arguments can be placed on 
additional lines. In order to choose the correct form the arguments 
must be rewritten and dimensioned and the available line width must be 
known. The arguments and symbols are concatenated horizontally until 
a line is filled. Successive lines are then concatenated vertically. 

In the case A + - B the connection symbol + can be omitted. 
Similiarly one might want to write A -v b, but A(B+C) , omitting the ^v 
when the parentheses remove the ambiguity. On the other hand, when the 

expression is continued on the next line one would still write- 
A ,but A 

(B+C) -B' ^^^^ problem is resolved by associating with each 

connecting symbol lists of those operators which cause it to be sur- 

pressed. There is a list of these operators which surpress it from the 

left, and a list of those operators which surpress it from the right. 

In addition there is a list of operators which replace the connecting 

symbol when a new line is started. Before an argument is rewritten 

and dimensioned its main operator is checked against the above lists 

so that the connecting symbol may be omitted if necessary. 

6. Title . 

The title operator takes as argimients an expression and its name. 
The name is rewritten, dimensioned, and positioned on the left edge 
of the display; the expression must then fit into the remaining space. 

7. Hybrid . 

Some operators are simply combinations of the others. 

In conclusion, rewriting is controlled by the available space, the 
type of operator, and the size and type of its arguments. The form 
is dependent on the dimensions only if line width is applied as a 
constraint . 
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III. Dimensions . 

The dimension pass starts at the ends of the branches of the in- 
ternal tree and works toward the base. At the ends of the branches 
are the individual variable names. These are either the names of 
special characters such as , or a sequence of characters to be 
displayed by the character generator. The names of special characters 
are marked with a flag and have their dimensions associated with them. 
The width of sequences of generated characters must be computed using 
their size and the fact that in the SAL language used for display * 
and = are mapped into character generator case shifts and / is 
used as a quote character, and also the fact that some generated char- 
acters are non-spacing. 

Five dimension functions must be written for each form. There are 
functions for the width, height, depth, symbols, and arguments. The 
compiler uses these functions to put the dimensions of each picture part 
on its property list. The three functions for the width, height, and depth 
have a list of the dimensioned arguments of the form as input. They 
use helping functions to retrieve the dimensions from the property 
lists of the arguments. The position of an argument or sjrmbol is 
specified by the coordinates of the lower left hand corner of the rec- 
tangle containing it. The function for the arguments has as value an 

S-expression of the form (^ X, A Y, tXx Ay ) where A X , 

l^j^ 1 n n^ i' 

■<^-\ Yi is the position of the i argument with respect to the lower 

left hand corner of the circumscribed rectangle. The function for the 

symbols has as output an S-expression of the form (A X. A Y NAME 

S^ W^ A X^ Ay^ NAME^ S^ W^) where the i*^^ symbol which is added 

to the display by this form has name NAME., size S., width W., and 
relative position /\ X^^ Y^. The functions for the symbols and argu- 
ments have as input the width, height, and depth, as well as a list 
of the dimensioned arguments. The compution of the dimensions and 
relative positions by independent functions breaks down when symbols 
such as parentheses must be chosen, as it is not desirable to repeat 
this choice for each of the five functions. To solve tliis problem 
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a mechanism is set up by which the width function, which is executed 
first, can communicate its choice to the others. 

Each of the forms currently in the system is pictured in the 
following figures. Dotted lines have been dra«, to Indicate how the 
arguments and center lines are placed. It might be well to note that 
the human eye is sensitive to even the smallest misalignments. 
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Figure 1: CONCAT 
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Figure 2: SUPSUB 





Figure 3: EVL 
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Figure 4: SUM 
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Figure 5: FAREN 



Figure 6: LVCONCAT 





Figure 7: TITLE 
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Figure 8: VCONCAT 
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Figure 9: DVD 
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Figure 10: ITG 



Figure 11: ABS 
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IV. Output to the Disp lay 

The LISP picture compiler program runs in Project MAC time sharing. 
After an expression has been dimensioned, the compiler outputs a 
description of the picture to be displayed over the dataphone to 
PDP-6 LISP. The picture Is displayed on the PDP-6 scope using the 
SAL and MACROSAL languages. Each expression picture is set up as a 
single MACROSAL object with the picture tree marked by left and right 
pseudo-parentheses. The compiler starts at the base of the tree and 
goes out the branches, taking them In left to right order. The 
compiler sends each symbol to the display as it is encountered. 
If two successive symbols are not adjacent, then the compiler links 
them with a non-displaying relative vector. Whenever the compiler encounters 
a new node of the Internal tree it follows the last symbol sent with a 
left pseudo-parenthesis. After the compiler has finished this node 
and all branches extending from it the corresponding right pseudo- 
parenthesis is sent. This information is also sent to the disk so 
that the picture can be reconstructed without being recompiled. 



A. I. Memo 85 -18- 



Memo rand urn MAC-M-257 



V. Example 

The following figure is a photograph of two displayed expressions. 
The source expression for El is: 

(EQN(PWR(NAM 0(R OMEGA NIL)NIL)*A NIL) 

(PLS(PRD(PRD 2 *L(PWR PI -1 NIL)NIL) 

(Pm)(*IX)G OMEGA NIL) (PWR(PLS *A 1 NIL)-1 NIL)NIL)NIL) 

(PRD(PRD 2 *A OMEGA (PWR PI -1 NIL) (PWR(*LOG OMEGA NIL) (PRD -1 *a NIL)NIL)NIL) 

(ITG *T PI (PRD(NAM 1(THETA *T NIL)NIL) 

(NAM(PLS *A -1 NIL) (M(PRD OMEGA 'Vr NIL)NIL)NIL)NIL)NIL)NIL)NIL)NIL) 

The source expression for E2 is: 

(0(SUM *V(PRD *N(PWR 2-1 NIL) NIL) 

(PLS *N -2 NIL) (PRD(ABS(NAM(PLS *v 1 NIL)B NIL)NIL) 

(PRD(PLS(PRD(PLS *N(PRD -1 *V NIL)NIL) 

(PLS(NAM(PLS *N(PRD -1 *V NIL)-1 NIL) P NIL) 

(PRD -1(NAM(PLS 'VN(PRD -1 *V NIL)-2 NIL)P NIL)NIL)NIL) 

NIL)(NAM(PLS( *N(PRD -1 *v NIL)-2 NIL)P NIL)NIL) 

(*LOG(PLS( *V 2 NIL)NIL)(NAM(PLS *V 1 NIL) 

THETA NIL)(PWR(PLS *V 1 NIL)-2 NIL) 

(PWR(PLS -m (PRD -1 *V NIL)NIL)-1 NIL)NIL)NIL)NIL)NIL) 



-19- 
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VI. Conclusion 

One measure of program complexity is the extent and predictability 
of the combinations of independent parts of the input data which must 
be made in order to determine the flow of program control. In this 
respect the display of mathematical expressions seems to ] ie midway 
in difficulty between the display of lines of english text and the 
display of arbitrary graphs. In the display of english text, the 
data is organized in a string, the only global property required is a 
character count. Hyphenation decisions are based on the word at the 
end of the line. On the other hand, the display of graphs seems to 
require the simultaneous positioning of several nodes, and might 
require a character algebra or an iterative approach. In the display 
of mathematical expressions, decisions can be made at each node of 
the expression tree, based on information collected from above and 
below. The central importance of this tree structure makes LISP a 
convenient language for the program. 
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Bibliography 

The Idea of rewriting a mathematical expression in terms of 
position operators and computing the dimensions recursively is con- 
tained in the memo by Minsky. He also suggests that the picture 
symbols should be parsed for Light pen reference, m his thesis 
Krakauer contributed several ideas about picture syntax 
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